A good question. Sims are difficult to validate for a number of reasons:
a) Vehicle simulation is complicated and detailed, so there are lots of places and ways to get things wrong, or where there is no right, do things differently
b) Most people haven't driven most of the cars available in most sims, in real life, to make for the best comparison
c) Getting all the required data, at sufficient accuracy, to model a vehicle in the simulation, is arguably hardest of all (without having the real car in front of you), so even with a perfect simulation, the numbers fed in can still throw the handling off (GIGO). Working on the assumption that you will never get all the numbers you need, it all depends on how well the rest of the numbers were estimated.
d) Vehicles interact with the terrain, and modelling the terrain accurately for a completely fair comparison is another large headache in itself, one that has been tackled far less.
When you consider that any objective tests you can do (skidpan for outright grip, acceleration and top speed testing, making sure locked wheels don't steer the vehicle, suspension moves in a correct looking manner, throttle has the correct response, and so on), will only ever cover any one basic aspect of physics (at a time), it means you can pass all such tests, and not really have a good idea of whether the sim is accurate or not.